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ABSTRACT 

The Community of Inquiry model provides a framework for recognizing and evaluating interpersonal 
behaviors in online educational settings. One of its three components, teaching presence, describes 
behaviors that are under the auspices of the online instructor. By examining behaviors through the 
theoretical lens provided by teaching presence, and by measuring them with the Teaching Presence Scale 
(TPS), it may be possible to better understand the most effective online instruction practices. The purpose 
of the study was to confirm the factor composition of TPS in an online professional development course 
and to determine the relationship between teaching presence and student satisfaction. Participants (n = 
718) were in-service educators enrolled in online professional development. Confirmatory factor analysis 
results provided strong support for a three-factor TPA model using 17 of the 28 original TPS items and 
evidence of a strong relationship between TPS and student satisfaction. The implications for practice 
center on how enhanced knowledge of teaching presence may be used to develop instructors or 
facilitators as online educators of adults. It is important that such instructors have a solid knowledge base 
in their field, as well as knowledge of and experience with andragogy. 
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I. INTRODUCTION 

In recent years, online education has become increasingly prevalent in American education [1,2]. This is 
true in higher education, corporate training, military settings, and K-12 education [3, 4, 5]. Online 
education has become a popular supplement to and replacement for traditional, face-to-face instruction, 
offering a cost effective and remotely accessible means of instruction to students previously lacking such 
opportunity [3, 4, 6]. 

While some principles and components of good online instruction are similar to those in face-to-face 
settings, there are substantial differences that need continued research [7]. Topics include expressing 
oneself and understanding others effectively without body language; learning how to project personality, 
or “social presence,” into online communication; projecting “teaching presence”; helping and interacting 
with students asynchronously; facilitating learning; and fostering a positive sense of community online. 
These aspects related to the construction of a sense of learning community are not easily achieved. They 
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are, however, of arguable importance in both online and face-to-face learning environments—Vygotsky 
[8] proposed that external social interactions become the basis for internal thought processes, and that 
learning is therefore inherently tied to learners’ social and cultural experiences. Specifically, he argued 
that people learn through collaboration with peers and others who possess greater levels of competence; 
the more competent assist the less competent through the learning process [9]. 

Other well-known learning and development theorists agreed that peer and student-teacher interaction 
play a powerful role in learning. While Piaget emphasized internal construction of knowledge more than 
Vygotsky, he acknowledged the role of peer interaction in the process [10]. Bandura also examined the 
impact of social interaction in learning and developmental progress, and concluded that people tend to 
learn largely through observation and “cognitively active” imitation of social role models [10, p. 64]. He 
also advocated a model of bidirectional reciprocation in learning. This was a radical departure from 
behaviorist models based on “unidirectional stimulus-response connections,” placing emphasis on the 
importance of community interactions and mutual learning experiences [10, p. 65]. Each of these three 
learning theorists agreed, albeit in different ways, that human learning is fostered through personal 
interaction, especially when these interactions take place with the assistance of more competent or 
knowledgeable leaders. 

II. THE COMMUNITY OF INQUIRY 

Acknowledging the importance of interaction in the learning process, Garrison, Anderson, and Archer 
attempted to explain how the social factors critical to human learning (e.g., peer interaction and the 
presence of a guiding/supporting/challenging role model) can be found in fully online, computer-based 
educational experiences [11], They proposed a “Community of Inquiry” conceptual framework to help 
identify the relationships between components that are required for learners to learn in online 
environments. This framework consists of cognitive presence, social presence, and teaching presence— 
educational elements that overlap and combine to comprise a total educational experience within an 
approach that is collaborative in nature. 

Cognitive presence is defined as “the extent to which learners are able to construct and confirm meaning 
through sustained reflection and discourse in a critical Community of Inquiry” [11, p. 11]. It is one crucial 
component of high-level thinking and learning. 

Social presence originates from telecommunications research performed in the 1970s in order to 
determine the degree to which one is able to project one’s personality into an experience and to interact 
with others’ personalities in audio and visual media like facsimile machines, voice mail, and audio 
teleconferencing [12. 13]. Social presence has been updated and further defined by Garrison et al. as “the 
ability of participants in a Community of Inquiry to project themselves socially and emotionally, as ‘real’ 
people (i.e., their full personality), through the medium of communication being used,” particularly in the 
medium of computer-mediated communication [14, p. 94]. Garrison et al. [14] point out the inherent 
difficulties in projecting one’s personality into online educational settings consisting entirely of written 
communication. Unlike in traditional, face-to-face settings, it is impossible in this type of online setting to 
read body language or listen to vocal inflections to achieve the deep levels of communication usually 
associated with social interactions. Forming a true Community of Inquiry in an online classroom is then 
challenging if social presence among students is weak, for it is these kinds of interactions that create a 
sense of community among learners. 

Teaching presence is the third component of the Community of Inquiry model. In the early days of online 
education, it was determined that there existed just as much need for a facilitator to design, direct, and 
inform the learning experience online as typically required in traditional, face-to-face classrooms [15]. 
Teaching presence is used to define and describe aspects of this facilitator’s role. It is “the design, 
facilitation and direction of cognitive and social processes for the purpose of realizing personally 
meaningful and educationally worthwhile learning outcomes” [16, p. 5], So, while cognitive presence 
describes learners’ higher-order thinking and learning in an online environment, and social presence 
refers to the projection of and interaction between their personalities, teaching presence shows how a 
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leader-facilitator helps to promote the most effective combinations of the two in order to bring about 
desired learning outcomes. It is this third component of the Community of Inquiry, and the measurement 
thereof, that is the focus of this study. 

A. Development and Validation of the Teaching Presence Scale 

Teaching presence has received increasing attention from researchers in recent years. Shea, Fredericksen, 
Pickett, and Pelz [17] took the teaching presence framework and qualitative coding scheme introduced by 
Anderson et al. [16] and created a survey to measure student perception of teaching presence in online 
courses. The Teaching Presence Scale (TPS) reflects the three factors of teaching presence originally 
proposed by Anderson et al. [16]: instructional design and organization, facilitating discourse, and direct 
instruction [17]. The instructional design and organization component breaks down into several 
subcategories of teaching presence: “setting the curriculum, establishing time parameters, utilizing the 
medium effectively, establishing netiquette, and designing instructional methods effectively” [17, p. 69]. 
Facilitating discourse includes “identifying areas of agreement and disagreement in online discussions, 
seeking to reach consensus, reinforcing student contributions, setting the climate for learning, drawing in 
other participants and prompting discussion, and assessing the efficacy of the process” (p. 70). Direct 
instruction includes “presenting content and questions, focusing the discussion on specific issues, 
confirming understanding, diagnosing misconceptions, and injecting knowledge from diverse sources” (p. 
71). 

The 28 items that make up the TPS reflect these three teaching presence factors (instructional design and 
organization, facilitating discourse, and direct instruction) and their subcategories. Each item was written 
in collaboration with the principal author of the teaching presence construct [16, 17], There was no 
apparent quantitative exploration of the factor structure represented by the survey items in initial studies 
using the TPS. Rather, the TPS appears to have been constructed via discourse between Shea et al. [17] 
and Anderson, one of the original framers of the teaching presence concept. The TPS was then used in at 
least two studies without quantitative validation of its results through factor analysis [17, 18]. 

The factor structure of the TPS was later explored using principal component analysis [19]. The sample 
(»=T067) of survey respondents was taken from a pool of online undergraduate students across 32 college 
campuses in the SUNY system. The results indicated the teaching presence model consisted of two 
factors, not three. Instead of instructional design and organization, facilitating discourse, and direct 
instruction proposed by Anderson et al. [16], the sample data suggested a combination of facilitating 
discourse and direct instruction. Shea [19] referred to this factor as “directed facilitation,” and stated it, 
along with the instructional design and organization factor, might make a more accurate two-factor model 
of teaching presence than the three factors originally proposed. 

Arbaugh and Flwang [20] also studied the factor structure of the three-factor teaching presence model 
proposed by Anderson et al. [16]. They sampled 191 MBA students using the survey created by Shea et 
al. [17]. Unlike the results found by Shea et al. [19], the confirmatory factor analysis conducted by 
Arbaugh and Flwang [20] supported the validity of the three-factor model when four poorly-fitting items 
were dropped from the analysis. 

Garrison [21], one of the original authors of the Community of Inquiry and teaching presence models, 
suggested one reason for the discrepancy could be the different samples. The sample studied by Shea et 
al. [19] was undergraduate students, while those studied by Arbaugh and Hwang [20] were graduate 
students. Garrison [21] hypothesized that graduate students who tend to be older and more experienced 
with higher education might have been more aware of the subtle distinctions between facilitating 
discourse and direct instruction as compared to undergraduate students. He suggested further analysis of 
the factors comprising the teaching presence model should be conducted in order to illuminate this 
phenomenon, and that there would be value in testing the factor structure in different settings and on 
different populations. It is this suggestion that leads, in part, to the purpose of the current study. 
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B. Purpose of the Study and Research Questions 

In many online education programs, student satisfaction is a primary course effectiveness evaluation tool 
[22, 23]. While it may be important to gauge student reactions as one measure of online teaching 
effectiveness, using this subjective tool as a primary basis for evaluation and decision making may be 
both inaccurate and imprecise [24]. Instead, more objective criteria are needed for evaluating the 
effectiveness of online courses and their instructors. These criteria do exist, and have been compiled into 
a survey instrument, the TPS. 

Additional evidence is needed to support the TPS factor structure. Moreover, the factor structure of 
teaching presence needs to be confirmed in a setting outside of higher education—the only arena in which 
it has currently been tested. The confirmatory factor analysis analyzed in the current study allowed for a 
test of the fit of the existing model using data gathered from a new setting, professional development. 
Finally, the data gathered by the instrument was correlated with traditional measures of student 
satisfaction to determine whether teaching presence relates positively to learners’ online experiences in a 
professional development setting. 

The purpose of this study is thus: first, to validate the use of the Teaching Presence Scale in an online 
educational setting outside of the higher education context in which it was designed and tested; second, to 
confirm the factor composition among facilitators in an online professional development course; and to 
determine the extent and direction of the relationship between teaching presence and student satisfaction. 
As the prevalence of online education grows, not only is it interesting to gauge whether students perceive 
the online course experience to be beneficial, but it is also important to see if factors like those contained 
in the teaching presence model can be harnessed to increase student satisfaction. The ability to positively 
impact students’ cognitive and affective experiences within online courses has practical implications for 
various facets of online education, from helping to gauge the effectiveness of online instructors, to 
ensuring students’ continued interest in registering for an institution’s online courses. 

The research questions to be answered in this study include the following: 

1) Does the factor structure of “teaching presence” for teachers completing an online professional 
development program fit the original three-factor model of teaching presence proposed for use in 
higher education (specifically instructional design and organization, 6 items; direct instruction, 10 
items; and facilitating discourse, 12 items)? 

2) Is there a correlation between teaching presence, as measured by the Teaching Presence Scale, and 
student satisfaction? 

III. METHODS 

A. Study Setting and Participants 

The Florida Online Reading Professional Development (FOR-PD) is a nationally recognized reading 
professional development project designed to assist pre- and in-service teachers in K-12 education to 
improve their knowledge and understanding of scientifically-based reading research and instructional 
practices [25]. Initiated in 2002, FOR-PD has educated nearly 45,000 teachers, administrators, and other 
educators. FOR-PD crosses two educational contexts, professional development and K-12 education, and 
its students are certified and practicing, K-12 teachers, as well as university and community college 
students taking the course as a component of teacher certification programs or graduate studies. FOR-PD 
instructors (also known as facilitators) are also experienced K-12 teachers with professional records as 
literacy leaders in the classroom. Facilitators are selected through an application process and/or 
recommended by their school district and receive a seven-week online certification course as well as 
ongoing professional development related to effective online facilitation [26]. 

FOR-PD offers its online course in open enrollment, district, community college, and university sections. 
Because the goal of the current research is to test the use of the TPS in a professional development setting 
removed from higher education, the study was limited to open enrollment and district course sections. 
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Both were led by veteran in-service K-12 teacher facilitators, and were attended by in-service K-12 
teacher participants. Participation in district sections was coordinated by individual school districts, with 
each section created specifically for and limited to teachers from a given district [27]. 

Open enrollment sections are defined as sections of the course that do not limit enrollment to teachers 
from a specific school district. Therefore, participants in open enrollment sections may not have the 
existing connections to their FOR-PD section classmates as typically seen in district sections, but they are 
in-service teachers with similar purposes for taking the FOR-PD course [27]. Additionally, this study 
only included participants who were enrolled in and completed the FOR-PD course during the spring of 
2009. As non-completers of a course do not have the same course experience (e.g., duration and 
completeness), they were eliminated from the study. 

B. Instrumentation 

1. Teaching Presence Scale 

The Teaching Presence Scale consists of 28 items, grouped by three teaching presence factors and their 
subcategories, and uses a five point Likert scale (strongly agree to strongly disagree). The instructional 
design and organization factor includes six items—two addressing the “setting the curriculum” 
subcategory, and one each for the “designing methods,” “establishing time parameters,” “using the 
medium effectively,” and “establishing netiquette” subcategories [18, pp. 69-70]. The facilitating 
discourse factor includes 12 items; two apiece for each of six subcategories (“identifying areas of 
agreement and disagreement; seeking to reach consensus and understanding; encouraging, 
acknowledging, and reinforcing student contributions; setting the climate for learning; drawing in 
participants and prompting discussion; [and] assessing the efficacy of the process”) [18, p. 70]. The direct 
instruction factor is represented by 10 items; two apiece for each of the five subcategories (“presenting 
content and questions, focusing the discussion on specific issues, summarizing discussion, confirming 
understanding, diagnosing misperceptions, injecting knowledge from diverse sources, [and] responding to 
technical concerns”) [18, p. 71]. 

Each subcategory in facilitating discourse and direct instruction factors contains two items due to the 
authors’ goal of determining whether teaching presence takes place and is measurable not only between 
instructors and students, but also among students themselves [18]. In these cases, the two items are 
worded identically, except one asks the respondent to evaluate the instructor’s teaching presence 
behavior, and the other asks the respondent to evaluate the participants’ teaching presence behavior. 

Development of and Previous Reliability and Validity Evidence for the Teaching Presence Scale. The 
TPS was developed and piloted in 2002 [17]. Respondents who reported high levels of TPS instructional 
design and organization (the first of the three teaching presence factors) also reported high levels of 
satisfaction (r = .64) and learning (r = .59). The same results held true for the relationship between 
facilitating discourse (the second of the three teaching presence factors) and reported satisfaction (r = .64) 
and learning (r = .59). Additionally, respondents who perceived high levels of direct instruction (the third 
teaching presence factor) in their experience also indicated high levels of satisfaction (r = .64) and 
reported learning (r = .61). Similar correlates with student satisfaction have been found in other studies 
(Shea, Pickett, et al., 2003). Exploratory factor analysis provided construct validity evidence of a two- 
factor solution: a) instructional design and organization and b) directed facilitation (direct instruction and 
facilitating discourse) [19,28]. Cronbach’s alpha was reported to be .97 for the overall results of the TPS, 
.94 for instructional design, and .97 for directed facilitation [28] with similar coefficients reported in later 
studies [20, 28], 

Arbaugh and Elwang [20] conducted confirmatory factor analysis (CFA) of the teaching presence scale to 
test the fit of the results with the theoretical three-factor model. While four of the TPS’s items were 
removed because they fit poorly (3 items dropped from facilitating discourse and 1 dropped from direct 
instruction), the results did support the original three-factor model. Instructional design and organization 
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was positively related with facilitating discourse and was also positively related with direct instruction. 
Facilitating discourse was positively related with direct instruction [20]. 

2. Student Satisfaction 

The student satisfaction instrument which sought to measure students’ overall course satisfaction included 
24 items. The first five items asked for basic information about the participant’s experience (i.e., whether 
they completed the course, whether they had enrolled via their school district or open enrollment, why 
they enrolled in FOR-PD, whether it was their first attempt at FOR-PD, and what they thought of the pace 
of the course). The following seven items asked participants to rate the course on dimensions such as the 
content’s ability to help meet their classroom needs and increase their knowledge of scientifically-based 
reading research, the ease of navigation through the course, appropriateness of the length of assignments 
and of the course itself, the helpfulness of course content tools in facilitating classroom implementation of 
reading strategies, and the amount of comfort respondents gained in using the reading strategies taught 
throughout the course. These were all measured using a five point Likert scale (strongly agree to strongly 
disagree). The next set of ten items asked respondents to rate their facilitator along dimensions such as 
feedback provided, interest in participants’ learning, assessment of participants’ progress, expression of 
expectations for performance, availability to assist students, promptness of responses in online 
discussions, promptness of responses via email or course mail, respect and concern for students, 
facilitation of learning, and overall assessment of facilitator’s effectiveness. These items used a four point 
scale (poor to excellent). The last two items asked participants to rate the frequency of their facilitator’s 
participation in and quality of online discussions. 

C. Data Collection Procedures 

The entire population of spring 2009 district and open enrollment participants was surveyed (N = 1061). 
The TPS was administered in mid-May 2009 via a link in an email, with a reminder email sent one week 
later. Anonymity was preserved in the current study by letting respondents login and answer survey 
questions without identifying themselves. 

D. Missing Data and Exclusion of Cases 

Of the original population of 1061 participants, 836 (79%) responded. Of these, 4 (<0.5%) were excluded 
because they indicated that they had not completed the course. As the TPS is designed to capture end-of- 
course impressions of online course completers, and as the experience of a non-completer may be very 
different from that of a completer, non-completers were eliminated from the sample [17]. 

Of the cases remaining (n = 832), listwise deletion was used to exclude 114 additional participants (14%) 
due to missing data in one or more of the Teaching Presence Scale items used for the CFA. This resulted 
in a sample size of 718. The listwise deletion method was chosen based on several factors: (a) an 
assumption of randomness of the missing data, (b) all measured variables effectively serving as dependent 
variables [29], and (c) the sample size after listwise deletion remained large enough to satisfy sample size 
criteria for performing CFA. To satisfy the assumption of randomness of the missing data, the dataset was 
examined for clear patterns of missing responses. Each variable had missing responses, the percentage of 
which ranged from 0.36% to 1.5%. No single variable contained a discemable majority of missing data, 
which might have otherwise indicated a non-random reason for missing responses [30], Regarding sample 
size, Flair et al. [30] suggest that one part of deciding how much missing data one can delete is to consider 
the impact to the planned statistical analysis. According to Tabachnick and Fidell [29], a sample size is 
optimal for CFA if there are at least 10 cases per parameters estimated. As this analysis included 59 
parameters estimated, the sample size is optimal with at least 590 cases. Listwise deletion of cases with 
missing responses produced a sample size larger than that (n=718), which was therefore adequate for 
performing the planned analysis. It is upon this dataset that the results of the demographic analysis are 
based. There was minimal missing data on the satisfaction items, (3%), thus missing satisfaction values 
were replaced with the median. 
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IV RESULTS 


A. Participants 

Of the 718 participants in the sample, 151 (22%) were male, 542 (78%) were female, and 25 (4%) did not 
indicate their gender. Approximately 64% (n = 460) of respondents were White, followed by 16% (n = 
114) Hispanic, and 14% (n = 97) Black. There was a small percentage of respondents who were 
American Indian or Alaskan native and Asian or Pacific Islander ( n =15, 2%). About 5% of respondents 
(n = 32) did not specify their race. Participants’ ages were fairly evenly distributed across categories 
ranging from 22 to 65+ years, with the majority between 26 and 30 years old (17%; n = 116). There was 
an almost even split between participants who had obtained their degrees through a college of education 
(55%; n = 393) and the participants who came into teaching through another route (42%; n = 301) (about 
3% of participants did not respond to this question). The majority of respondents (42%; n = 298) had 
between two and five years of experience in K-12 education, and most (83%; n = 597) were regular full¬ 
time teachers. Many (30%; n = 213) were public high school teachers, followed closely by public middle 
school teachers (22%; n = 157). For a majority of respondents, (34%; n = 242), FOR-PD was their first 
online course experience. This was followed by those for whom it was their fifth online course (26%; n = 
187) and by those for whom it was their second online course (18%; n = 129). Of the total population 
(7V=718), 83% (n = 571) said they would consider taking additional online courses in the future. 

Pertaining to their FOR-PD course participation, most respondents (80%, n = 571) indicated they enrolled 
in the course through sections organized by their school districts, rather than through open enrollment. 
The primary reason most participants took the course was that it was a necessary part of obtaining a 
reading endorsement for their teaching certificate (50%, n = 356). Most participants (85%, n = 609) were 
enrolled in FOR-PD for the first time. However, some participants (13%, n = 91) were repeating it after 
having not completed it (about 3% of participants did not respond to this item). 


B. Hypothesized Model 

Confirmatory factor analysis was conducted using L1SREL 8.7. The hypothesized three-factor model is 
shown in Figure 1, including Instructional Design and Organization, Facilitating Discourse, and Direct 
Instruction as latent factors [16]. Teaching Presence Scale Items 1 through 6 are hypothesized to serve as 
indicators for Instructional Design and Organization. Items 7 through 18 are hypothesized to indicate 
Facilitating Discourse, and items 19 through 28 are hypothesized to indicate Direct Instruction. The three 
latent factors are hypothesized to covary with each other. 
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C. Data Screening 

Data (n= 718) were screened using univariate tests for normality. Muthen and Kaplan [31] and Curran, 
West, and Finch [32] determined through extensive simulation that CFA results begin to be distorted, and 
the likelihood of Type I error inflated, as univariate skewness and kurtosis values approach absolute 
values of 2 and 7, respectively. In the current study, skewness ranged from -.669 to -2.477, with four 
items (#1, 2, 3, and 4) slightly exceeding an absolute value of 2. Kurtosis values ranged from 0.205 to 
6.933, within acceptable range. Other researchers note there is little agreement on what constitutes 
acceptable levels of non-normality in CFA [33]. As skewness and kurtosis levels for this study were 
largely within parameters specified by Muthen and Kaplan [31] and Curran et al. [32], it was decided to 
proceed without making further modifications to the dataset. Maximum likelihood (ML) was the 
estimation method used [32, p. 17]. Normality is assumed with this estimation method, without which its 
use of the normal theory chi square statistic results in an inflated rate of model rejection and Type 1 error. 
This problem can be alleviated by the use of the Satorra-Bentler chi square statistic and other alternatives 
instead of the normal theory chi square statistic, so these fit indices were also examined. 

D. Research Question 1 Results 

The independence model that challenges the hypothesis that all variables are uncorrelated was rejected, £ 
(378, A = 718) = 150181.78. The hypothesized model was then tested, and indicators of model fit were 
examined. The chi-square goodness-of-fit index was statistically significant and therefore indicative of a 
poorly-fitting model, / 2 (347, A = 718) = 9167.46, p < .01, but reliance on this statistic is to be done with 
caution due to its susceptibility to non-normality and sample size [34]. As the current study has a 
moderately large sample (n = 718), and as the data did have some tendencies toward non-normality, a 
Satorra-Bentler scaled chi-square and a chi-square corrected for non-normality were examined, x 2 (347, 
A = 718) = 4454.58, p < .01 and % 2 (347, A = 718) = 407.48, p = .014, respectively. The latter may 
provide some indication of goodness-of-fit, although it is still statistically significant, as it is less than two 
times the model degrees of freedom [29]. See Table 1. 

Table 1. CFA models’ goodness of fit indices and acceptable levels 
Goodness Ancillary three- 

of fit Initial three-factor factor model Ancillary two-factor 

criterion model [17] statistic [28] model statistic [28] Acceptable level 

Chi-square j 2 (347, A = 718) = / 2 (116, A = 718) j 2 (118, A = 718) = Not statistically 

9167.46, p <.01 =980.29 ,p < .01 1226.25 ,p < .01 significant, 

and/or less than two 
times the model 
degrees of freedom 

Satorra- j 2 (347, A = 718) = / 2 (116, A = 718) j 2 (116, A = 718) = Not statistically 


Bentler chi- 
square 

4454.58, p<. 01 

= 174.08, p<. 01 

228.15, p<. 01 

significant, 

and/or less than two 
times the model 
degrees of freedom 

Chi-square 

j 2 (347, A =718) = 

j 2 (116, A =718) 

/ 2 (116,A= 718) = 

Not statistically 

adjusted for 
non- 

normality 

407.48, p = . 014 

= 115.56, p = . 49 

128.36, p=. 24 

significant, 

and/or less than two 
times the model 
degrees of freedom 

CF1 

.97 

.999 

.998 

>.95 

NNFI 

.97 

.999 

.998 

>.90 
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SRMR 

.09 

.02 

.02 

<.08 

RMSEA 

.13 

.03 

.04 

<.10 


The comparative fit index, or CFI, (.97) and non-normed fit index, or NNFI (.97) provided support for 
goodness of model fit, as values in excess of .95 and .90, respectively, are considered to be indicative of a 
well-fitting model [36, 30]. The standardized root mean square residual, or SRMR (.093) was less 
supportive of goodness-of-fit, as values less than .08 are desirable. The root mean square error of 
approximation, or RMSEA (.13) also failed to provide support, as values in excess of .10 imply a poor 
model fit (see Table 2). The fact that these fit indices were rather weak could indicate a poorly-fitting 
model. Thus ancillary analysis was explored, as detailed later. 

Table 1. Results of confirmatory factor analysis of teaching presence variables 

Unstandardized 


Measure and variable 

factor loading 

SE 

Uniqueness 

t 

TPS - Instructional Design and Organization 





Item 1: Overall, the instructor for this course 
clearly communicated important course 
outcomes. 

.96 

.05 

.09 

57.90 

Item 2: Overall, the instructor for this course 
clearly communicated important course 
topics. 

.96 

.05 

.08 

64.09 

Item 3: Overall, the instructor for this course 
provided clear instructions on how to 
participate in course learning activities. 

.95 

.04 

.11 

73.68 

Item 4: Overall, the instructor for this course 
clearly communicated important due 
dates/time frames for learning activities that 
helped me keep pace with the course. 

.94 

.04 

.12 

74.38 

Item 5: Overall, the instructor for this course 
helped me take advantage of the online 
environment to assist my learning. 

.93 

.05 

.13 

57.26 

Item 6: Overall, the instructor for this course 
helped student to understand and practice the 
kinds of behaviors acceptable in online 
learning environments. 

.90 

.05 

.18 

52.02 

TPS - Facilitating Discourse 





Item 7: Overall, the instructor for this course 
was helpful in identifying areas of agreement 
and disagreement on course topics that 
assisted me to learn. 

.90 

.04 

.18 

67.18 

Item 8: Overall, other participants in this 
course were helpful in identifying areas of 
agreement and disagreement on course topics 
that assisted me to learn. 

.76 

.05 

.42 

29.82 

Item 9: Overall, the instructor for this course 

.94 

.04 

.13 

97.51 


was helpful in guiding the class towards 
agreement/understanding about course topics 
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that assisted me to learn. 


Item 10: Overall, other participants in this 
course were helpful in guiding the class 
towards agreement/understanding about 
course topics that assisted me to learn. 

Item 11: Overall, the instructor in this course 

.78 

.05 

.40 

32.07 

acknowledged student participation in the 
course. 

.91 

.04 

.17 

72.67 

Item 12: Overall, other participants in this 
course acknowledged student participation in 
the course. 

.77 

.05 

.40 

30.67 

Item 13: Overall, the instructor for this course 
encouraged students to explore concepts in 
the course. 

.93 

.04 

.14 

74.25 

Item 14: Overall, other participants in this 
course encouraged students to explore 
concepts in the course. 

.78 

.05 

.40 

32.43 

Item 15: Overall, the instructor for this course 
helped to keep students engaged and 
participating in productive dialog. 

.94 

.04 

.12 

96.94 

Item 16: Overall, other participants in this 
course helped to keep students engaged and 
participating in productive dialog. 

.79 

.05 

.38 

33.53 

Item 17: Overall, the instructor for this course 
helped keep the participants on task in a way 
that assisted me to learn. 

.94 

.04 

.11 

125.49 

Item 18: Overall, other participants in this 
course helped keep us on task in a way that 
assisted me to learn. 

TPS - Direct Instruction 

.75 

.05 

.43 

28.91 

Item 19: Overall, the instructor for this course 
presented content or questions that helped me 
to learn. 

.94 

.05 

.12 

62.76 

Item 20: Overall, other participants in this 
course presented content or questions that 
helped me to learn. 

.76 

.05 

.42 

28.44 

Item 21: Overall, the instructor for this course 
helped to focus discussion on relevant issues 
in a way that assisted me to learn. 

Item 22: Overall, other participants in this 

.96 

.04 

.08 

132.05 

course helped to focus discussion on relevant 
issues in a way that assisted me to learn. 

.79 

.05 

.38 

32.53 

Item 23: Overall, the instructor for this course 
provided explanatory feedback that assisted 

.94 

.04 

.13 

101.38 


me to learn. 
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Item 24: Overall, other participants in this 
course provided explanatory feedback that 
assisted me to learn. 

.79 

.05 

.38 

31.22 

Item 25: Overall, the instructor for this course 
helped me to revise my thinking in a way that 
helped me to learn. 

.92 

.04 

.15 

91.69 

Item 26: Overall, other participants in this 
course helped me to revise my thinking in a 
way that helped me to learn. 

.76 

.05 

.43 

31.01 

Item 27: Overall, the instructor for this course 
provided useful information from a variety of 
sources that assisted me to learn. 

.90 

.05 

.20 

52.20 

Item 28: Overall, other participants in this 
course provided useful information from a 
variety of sources that assisted me to learn. 

.75 

.05 

.43 

29.82 


There were three distinct factors inherent within teaching presence, and they were shown to share a 
positive covariance. As shown in Figure 1, TPS items 1 through 6 loaded on the instructional design and 
organization factor. All path coefficients were statistically significant, indicating each item is indeed a 
statistically significant indicator of the latent construct of instructional design and organization. Factor 
loadings ranged from .90 to .96, indicating a very strong relationship between the TPS items and the 
latent teaching presence construct. Standard error of measurement values ranged from .04 to .05, and 
uniqueness (measurement error of the variance) values ranged from .08 to .18. These indicate that only 
very small amounts of the variance were attributable to phenomena other than the item’s ability to 
measure the latent factor. Additionally, r 1 values ranged from .82 to .92, meaning 82% to 92% of the 
score variance could be accounted for by these items’ ability to measure the teaching presence construct 
of instructional design and organization. 

TPS items 7 through 18 loaded on the facilitating discourse factor. Again, all path coefficients were 
statistically significant, indicating each item to be a statistically significant indicator of the facilitating 
discourse construct. Factor loadings ranged from .75 to .94, again indicating reasonably strong 
relationships (approaching a value of 1) between the TPS items and the latent teaching presence construct. 
Standard error of measurement values ranged from .04 to .05. Uniqueness (measurement error of 
variance) values ranged from .11 to .43. This indicated that for some of the items (especially 8, 10, 12, 14, 
16, and 18) there were increased amounts of the variance attributable to phenomena other than the items’ 
ability to measure the latent factor. The r 2 values ranged from .56 to .89, meaning 56% to 89% of the 
score variance could be accounted for by these items’ ability to measure the teaching presence construct 
of facilitating discourse. The r 2 values were lower for the same five even-numbered items that had greater 
uniqueness values, indicating these items may not measure facilitating discourse as strongly as do items 7, 
9, 11, 13, 15, and 17. This will be the subject of additional discussion later when ancillary models are 
presented. 

TPS items 19 through 28 loaded on the direct instruction factor. Once again, all path coefficients were 
statistically significant, indicating each item to be a statistically significant indicator of the direct 
instruction construct. Factor loadings ranged from .75 to .96, again indicating reasonably strong 
relationships (approaching a value of 1) between the TPS items and the latent teaching presence construct. 
Standard error of measurement values ranged from .04 to .05. Uniqueness (measurement error of 
variance) values ranged from .08 to .43. This indicated that for some of the items (especially 20, 22, 24, 
26, and 28) there were increased amounts of the variance attributable to phenomena other than the items’ 
ability to measure the latent factor. The r 2 values ranged from .57 to .92, indicating that 57% to 92% of 
the score variance could be accounted for by these items’ ability to measure the teaching presence 
construct of direct instruction. The r 2 values were lower for the same five even-numbered items that had 
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greater uniqueness values, indicating that these items may not measure direct instruction as strongly as do 
items 19, 21, 23, 25, and 27. This will be the subject of additional discussion later in this article when 
ancillary models are presented. 

E. Ancillary Analyses 

Because of the weakness of the initial model as indicated by statistically significant chi square goodness- 
of-fit indices and mixed results among other fit indices, ancillary analyses were generated to determine 
whether alternative teaching presence models, as presented by other researchers, might produce stronger 
evidence of model fit. Two alternative models were tested; a three-factor model using fewer items as 
proposed by Shea et al. [28] and a two-factor model proposed by Shea et al. [28] (instructional design and 
organization, and directed facilitation). 

1. Three-Factor Model without Participant Items (Shea et al., 2005) 

The first model explored in the ancillary analysis was proposed by Shea et al. [28] who modified their 
selection of TPS items in reaction to initial exploratory factor analysis. Instead of using the original 28- 
item TPS (with six items measuring instructional design and organization, 12 items measuring facilitating 
discourse, and 10 items measuring direct instruction) they used a modified TPS of 17 items which 
included six instructional design and organization items, six facilitating discourse items, and five direct 
instruction items. The latter two TPS components included all items pertaining to the instructor, but did 
not include those items pertaining to other participants. 

Although the rationale for excluding these items is not discussed in the article by Shea et al. [28], the 
results of the initial analysis in the current study seem to support this modification. It may be seen in 
Table 2 that the factor loadings and r values were all weaker for the participant-oriented items than for 
the instructor-oriented items. For example, the aforementioned Item 21, pertaining to the instructor’s 
helpfulness in focusing discussion on relevant issues, had a factor loading of .96 and an r 2 of .92. Its 
factor loading was statistically significant (t = 132.05). More than 92% of the score variance could be 
accounted for by this item’s ability to measure the teaching presence construct of direct instruction. On 
the other hand. Item 22, pertaining to the other participants’ helpfulness in focusing discussion on 
relevant issues, had a factor loading of .79 and an r" of .63. Its factor loading was also statistically 
significant (t = 32.53), but overall, it appeared to be a weaker measure of direct instruction than the item 
pertaining to the instructor’s performance. 

The CFA was therefore re-run using only the scores from items 1-6 (hypothesized to load on the 
instructional design and organization factor); items 7, 9, 11, 13, 15, and 17 (hypothesized to load on the 
facilitating discourse factor); and items 19, 21, 23, 25, and 27 (hypothesized to load on the direct 
instruction factor). As shown in Figure 2 and Table 3, the items in this first ancillary model loaded 
successfully on the intended factors. The independence model that challenges the hypothesis that all 
variables are uncorrelated was rejected, £ (136, N= 718) = 69807.78. The first ancillary model was then 
tested, and indicators of model fit were much improved as compared to the 28-item model, % 2 (116, N = 
718) = 980.29 ,p < .01; CF1 = .999; NNF1 = .999; SRMR = .02; and RMSEA = .03. 

The chi-square goodness-of-fit index was statistically significant and technically indicative of a poorly- 
fitting model, but as stated previously, reliance on this statistic is to be done with caution due to its 
susceptibility to non-normality and sample size [34]. As with the 28-item model, a Satorra-Bentler scaled 
chi-square and a chi-square corrected for non-normality were examined, j 2 (116, N = 718) = 174.08, p < 
.01 and / 2 (116 , N = 718) = 115.56, p = .49, respectively. The latter definitely provides indication of 
goodness-of-fit, as it is no longer statistically significant (p = .49), and as it is clearly less than two times 
the model degrees of freedom [29], The CF1 and NNF1 provided support for goodness of model fit [29]. 
The SRMR and RMSEA were also supportive of goodness-of-fit (see Table 2). (The correlation matrix is 
available upon request). 
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Table 2. Results of ancillary three-factor confirmatory factor analysis of teaching presence 

Unstandardized 


Measure and variable 

factor loading 

SE 

Uniqueness 

t 

TPS - Instructional Design and Organization 





Item 1: Overall, the instructor for this course 
clearly communicated important course 
outcomes. 

.96 

.05 

.09 

58.10 

Item 2: Overall, the instructor for this course 
clearly communicated important course 
topics. 

.96 

.05 

.08 

64.70 

Item 3: Overall, the instructor for this course 
provided clear instructions on how to 
participate in course learning activities. 

.95 

.04 

.11 

74.36 

Item 4: Overall, the instructor for this course 
clearly communicated important due 
dates/time frames for learning activities that 
helped me keep pace with the course. 

.94 

.04 

.12 

74.89 

Item 5: Overall, the instructor for this course 
helped me take advantage of the online 
environment to assist my learning. 

.93 

.05 

.14 

57.61 

Item 6: Overall, the instructor for this course 
helped student to understand and practice the 
kinds of behaviors acceptable in online 
learning environments. 

.90 

.05 

.19 

52.16 

TPS - Facilitating Discourse 





Item 7: Overall, the instructor for this course 
was helpful in identifying areas of agreement 
and disagreement on course topics that 
assisted me to learn. 

.91 

.04 

.17 

68.86 

Item 9: Overall, the instructor for this course 
was helpful in guiding the class towards 
agreement/understanding about course topics 
that assisted me to learn. 

.94 

.04 

.12 

89.95 

Item 11: Overall, the instructor in this course 
acknowledged student participation in the 

.93 

.04 

.15 

77.60 

course. 





Item 13: Overall, the instructor for this course 
encouraged students to explore concepts in 
the course. 

.93 

.05 

.14 

67.01 

Item 15: Overall, the instructor for this course 
helped to keep students engaged and 
participating in productive dialog. 

.94 

.04 

.11 

95.75 

Item 17: Overall, the instructor for this course 

.95 

.04 

.10 

122.30 


helped keep the participants on task in a way 
that assisted me to learn. 

TPS - Direct Instruction 
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Item 19: Overall, the instructor for this course 
presented content or questions that helped me 
to learn. 

.94 

.05 

.11 

59.03 

Item 21: Overall, the instructor for this course 
helped to focus discussion on relevant issues 
in a way that assisted me to learn. 

.97 

.04 

.07 

127.32 

Item 23: Overall, the instructor for this course 
provided explanatory feedback that assisted 
me to learn. 

.95 

.04 

.10 

109.84 

Item 25: Overall, the instructor for this course 
helped me to revise my thinking in a way that 
helped me to learn. 

.92 

.04 

.15 

81.10 

Item 27: Overall, the instructor for this course 

.91 

.05 

.18 

53.87 


provided useful information from a variety of 
sources that assisted me to learn. 

TPS items 1 through 6 loaded on the instructional design and organization factor. All path coefficients 
were statistically significant, ranging from .90 to .96 (and possessing t test statistic values ranging from 
52.16 to 74.89), standard error of measurement values ranging from .04 to .05, and measurement error of 
variance values ranging from .08 to .19. Additionally, r 2 values ranged from .82 to .92. 

TPS items 7, 9, 11, 13, 15, and 17 all loaded on the facilitating discourse factor. Again, all path 
coefficients were statistically significant, ranging from .91 to .95 (and possessing t test statistic values 
ranging from 67.01 to 122.30), standard error of measurement values ranging from .04 to .05, 
measurement error of variance values ranging from .10 to .17. In addition, r 1 values ranged from .83 to 
.90. 

TPS items 19, 21, 23, 25, and 27 all loaded on the direct instruction factor. All path coefficients were 
statistically significant, ranging from .91 to .97 (and possessing t test statistic values ranging from 53.87 
to 127.32), standard error of measurement values ranging from .04 to .05, and measurement error of 
variance values ranging from .07 to .17. Additionally, r 2 values ranged from .81 to .93. 
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Figure 2. Three-factor 17-item model (Shea et al., 2005) 


Internal consistency of the 17-item TPS model was examined using Cronbach’s alpha, and produced a 
value of .98. A substantial increase in this Cronbach’s alpha value would not be achieved by deleting any 
items from the scale. 

2. A Two-Factor Model of Teaching Presence [29] 

The second model explored in the current ancillary analysis was also proposed by Shea et al. [28] who 
found that certain items (1 through 5) loaded on the instructional design and organization factor, as 
anticipated. However, other items loaded on one factor that seemed to combine facilitating discourse and 
direct instruction. This new factor was termed “directed facilitation.” For brevity and because the two- 
factor model was ruled out as the strongest of the three tested in the current study, only model fit statistics 
are presented (see Table 2). Comprehensive results are available upon request from the author. 
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3. Summary of Model Fit in Ancillary Analyses 

Although both ancillary models showed evidence of a better fit than the original model (28 items, three 
factors), the first ancillary model (17 items, three factors) was slightly better-fitting than the second model 
(17 items, two factors) (see Table 2). As a result, composite scores created from the first ancillary model 
were correlated with the student satisfaction items. 

F. Research Question 2 Results 

The second question asked, is there a correlation between teaching presence, as measured by the Teaching 
Presence Scale, and student satisfaction? Prior to addressing this question, evidence of construct validity 
and reliability for the satisfaction scale is provided. 

1. Validity and Reliability Analysis of Student Satisfaction Items 

Seventeen items were used to measure student satisfaction. These items asked participants to rate the 
course on dimensions such as the content’s ability to help meet their classroom needs and increase their 
knowledge of scientifically-based reading research, the ease of navigation through the course, 
appropriateness of the length of assignments and of the course itself, the helpfulness of course content 
tools in facilitating classroom implementation of reading strategies, and the amount of comfort 
respondents gained in using the reading strategies taught throughout the course. Items were measured 
using a five point Likert scale (strongly agree to strongly disagree). 

Ten items asked respondents to rate their FOR-PD facilitator along dimensions such as feedback 
provided, interest in participants’ learning, assessment of participants’ progress, expression of 
expectations for performance, availability to assist students, promptness of responses in online 
discussions, promptness of responses via email or course mail, respect and concern for students, 
facilitation of learning, and overall assessment of facilitator’s effectiveness. These items used a four point 
scale (“poor,” “fair,” “good,” and “excellent”). 

Prior to correlating the TPS scores with the student satisfaction scores, an exploratory factor analysis 
(EFA) was conducted. It was anticipated there would be two factors, one for satisfaction with the FOR- 
PD course and one for satisfaction with FOR-PD facilitators. Because the response scale for the two sets 
of items differed (i.e., strongly agree to strongly disagree and excellent to poor), EFA was generated 
independently for each set of items. The first step in determining the factorability of course satisfaction 
items was to review the communalities. There were no communalities above 1.0, so all seven items were 
retained for analysis. The initial factorability of these items was examined using common criteria for 
determining factorability including: (a) reviewing correlation of items; (b) Kaiser-Meyer-Olkin measure 
of sampling adequacy (overall and individual); (c) Bartlett’s test of sphericity; and (d) communalities. 
Regarding (a) the correlation of the items (available upon request), all seven items correlated greater than 
.30 with at least one other item, and all seven were statistically significant {p < .001). Regarding (b), the 
Kaiser-Meyer-Olkin measure of sampling adequacy was .86, larger than the recommended value of .50. 
The measure of sampling adequacy values for the individual items were all .78 or above, again proving to 
be larger than the recommended value of .50. Regarding (c), Bartlett’s test of sphericity was statistically 
significant \j? (21) = 2719.43, p < .001]. Regarding (d) communalities, all seven items had communality 
values in excess of the recommended value of .30 (see Table 4). This provides evidence of shared 
variance among the items. As these four common criteria for determining factorability were all easily 
met, it was deemed reasonable to proceed with determining the factor structure of the seven items. 
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Table 4. Factor Loadings and Communalities Based on Maximum Likelihood Analysis for Satisfaction with Course and 


Satisfaction with Facilitator Items 31 ■ 

- 40 (n = 718) 


Item 

Factor 1 

Communality 

Satisfaction with Course Items 



1 found the FOR-PD course content met my needs on 
learning how to integrate literacy into my content area 
instruction. 

.85 

.72 

The course content increased my knowledge of 
scientifically-based reading research. 

.84 

.70 

It was easy to navigate through the course. 

.69 

.48 

The assignments were of an appropriate length (not too 
time consuming). 

.62 

.39 

The course in general was of an appropriate length. 

.69 

.48 

The literacy log was helpful for classroom 
implementation of strategies. 

.70 

.49 

1 am comfortable using the reading strategies taught in 
FOR-PD. 

.70 

.49 

Satisfaction with Facilitator Items 

Rate the feedback concerning your performance in this 
course. 

.86 

.75 

Rate the facilitator's interest in your learning. 

.91 

.82 

Rate the facilitator's assessment of your progress in the 
course. 

.89 

.80 

Rate the facilitator’s expression of expectations for 
performance. 

.86 

.73 

Rate the facilitator’s availability to assist students. 

.88 

.77 

Rate the promptness of facilitator responses in online 
discussions. 

.87 

.75 

Rate the promptness of facilitator responses in email or 
course mail. 

.85 

.72 

Rate the facilitator’s respect and concern for students. 

.85 

.72 

Rate the facilitator’s facilitation of learning. 

.91 

.84 

Overall assessment of facilitator. 

.94 

.88 


The maximum likelihood estimation procedure with promax rotation was used to extract the factors. 
Initial eigenvalues indicated the first factor explained approximately 61% of the variance. The remaining 
factors did not have eigenvalues greater than one, so solutions for more than one factor were not 
examined. The one-factor solution, which represented 53% of the variance explained when extracted, was 
preferred due to theoretical support, review of the scree plot which indicated that the eigenvalues leveled 
off after one factor, and difficulty in interpreting two or more factors. All items contributed to a simple 
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factor structure and had a primary factor ranging from .62 to .85, well above the recommended value of 
.30. Table 5 contains the factor loading pattern matrix for this final solution. The name of the factor is 
Satisfaction with Course. The results of the factor analysis lend support to internal structure validity 
evidence supporting the conclusion that the scores produced by these seven items are a valid assessment 
of participants’ satisfaction with the FOR-PD course. Internal consistency was examined using 
Cronbach’s alpha, and produced a value of .88. A substantial increase in this Cronbach’s alpha value 
would not be achieved by deleting any items from the scale. Finally, a composite score was created for 
the factor by computing the mean of the seven items making up the scale. This composite score was then 
used as part of a Pearson correlation with TPS scores and other student satisfaction scores, which will be 
presented later. 

An EFA was conducted to determine evidence of the factor structure of the Satisfaction with the FOR-PD 
Facilitators items (10 items). The first step was review of the communalities. None were above 1.0, so all 
10 items were retained for analysis. The initial factorability of these items was examined, using common 
criteria for determining factorability as indicated previously. Regarding (a) the correlation of the items, 
all 10 items correlated greater than .30 with at least one other item, and all 10 were statistically significant 
(p < .001) (correlation table is available upon request). Regarding (b), the Kaiser-Meyer-Olkin measure of 
sampling adequacy was .96, comfortably larger than the recommended value of .50. The measure of 
sampling adequacy values for the individual items were all .94 or above, again proving to be larger than 
the recommended value of .50. Regarding (c), Bartlett’s test of sphericity was statistically significant 
(45) = 8866.71,< .001]. Regarding (d) communalities, all 10 items had communality values in excess of 
the recommended value of .30. This may be seen in Table 5. This provides evidence of shared variance 
among the items. As these four common criteria for determining factorability were all easily met, it was 
deemed reasonable to proceed with determining the factor structure of the 10 satisfaction with facilitator 
items. 

The maximum likelihood estimation procedure with promax rotation was used to extract the factors. 
Initial eigenvalues indicated that the first factor explained about 80% of the variance when extracted. The 
remaining factors did not have eigenvalues greater than one, so solutions for more than one factor were 
not examined. The one-factor solution was preferred due to theoretical support, review of the scree plot 
which indicated that the eigenvalues leveled off after one factor, and difficulty in interpreting two or more 
factors. 

All items contributed to a simple factor structure and had a primary factor ranging from .85 to .94, well 
above the recommended value of .30. Table 5 contains the factor loading pattern matrix. The name of the 
factor is Satisfaction with Facilitator. The results of the factor analysis lend support to internal structure 
validity evidence supporting the conclusion that the scores produced by these items are a valid assessment 
of participants’ satisfaction with the facilitators. Internal consistency for these scores was examined using 
Cronbach’s alpha and was .97. A substantial increase in this Cronbach’s alpha value would not be 
achieved by deleting any items from the scale. Finally, a composite score was created for the factor by 
computing the mean of the ten items making up the scale. 

Correlation coefficients were computed to determine if there were relationships among the three TPS 
composite variables (instructional design and organization, facilitating discourse, and direct instruction) 
and the two student satisfaction composite variables (satisfaction with FOR-PD course, satisfaction with 
FOR-PD facilitators). Review of a scatterplot of the variables suggested linear relationships between the 
variables were feasible, so the decision was made to continue with the correlation procedure. 

The results of the correlational analyses indicate all of correlation coefficients were statistically 
significant (p < .01). For the most part, the results indicate positive and moderate to strong relationships 
between the three teaching presence composite variables and the two student satisfaction composite 
variables, where moderate correlation is defined by a coefficient ranging from .50 to .70, and strong 
correlation is defined by a coefficient ranging from .70 to .90 [36]. As participants’ sense of teaching 
presence moves in a positive direction, so does their feeling of satisfaction with both the course and the 
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facilitator. The correlation coefficients can be seen in Table 5. The strongest relationships were seen 
between direct instruction and satisfaction with facilitators, r = .73, p < .01, facilitating discourse and 
satisfaction with facilitators, r = .73, p < .001, and instructional design and organization with satisfaction 
with facilitators, r = .65, p < .01. Relationships between teaching presence factors and satisfaction with 
the course were moderate and statistically significant, with direct instruction relating most strongly with 
satisfaction with the course, r = .59, p < .01. This was followed by the relationship between facilitating 
discourse and satisfaction with the course, r = .56, p < .01, and between instructional design and 
organization and satisfaction with the course, r = .52, p < .01. Shared variance between all of these 
variables was greater than 25%, generally interpreted to be large effects [37]. 


Table 5. Correlations Between Teaching Presence and Student Satisfaction (it = 718) 


Teaching 

Teaching Presence: Presence: 
Instructional Design Facilitating 
Subscale & Organization Discourse 

Teaching 

Presence: 

Direct Instruction 

Satisfaction with 
the Course 

Teaching Presence: gg* 

Facilitating Discourse 

— 



Teaching Presence: g 

Direct Instruction 

.94* 

— 


Student Satisfaction: 

Satisfaction with the .52* 

Course 

.56* 

.59* 

— 

Student Satisfaction: 

Satisfaction with the .65* 

Facilitator 

.73* 

.73* 

.43* 


*p < .01 


V. DISCUSSION 


As defined by Garrison et al. [11], teaching presence is the component of the Community of Inquiry 
model that discusses the behaviors and functions of an online instructor or facilitator. Garrison et al. [11] 
presented teaching presence as comprised of three constructs: instructional design and organization, 
facilitating discourse, and direct instruction. These three factors were represented in the initial validation 
of the Teaching Presence Scale [17]. Exploratory factor analysis led Shea et al. [28] to propose a different 
structure for teaching presence. Instead of the three factors presented by Garrison et al. [11], the analysis 
performed by Shea et al. [28] supported a two-factor model, combining facilitating discourse and direct 
instruction into a single factor called directed facilitation. A confirmatory factor analysis by Arbaugh and 
Hwang [20], on the other hand, found support for the original three-factor model. Although these 
researchers found divergent evidence of the teaching presence factor structure, they found the TPS items 
loaded more effectively as valid measures of teaching presence when items measuring teaching presence 
behaviors among participants in the course were excluded from the analysis. It was with this research 
history in mind that the initial and ancillary analyses were conducted in the current research study. 

The CFA first tested in this study was the original, three-factor model (including instructional design and 
organization, facilitating discourse, and direct instruction as three distinct constructs) with all original 28 
TPS items used as variables. There were six items measuring instructional design and organization, 12 
items measuring facilitating discourse, and 10 items measuring direct instruction. As mentioned 
previously, facilitating discourse and direct instruction were measured for both instructors and students. 
In other words, each item measuring an instructor’s facilitating discourse or direct instruction behaviors 
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was duplicated to also measure participants’ facilitating discourse or direct instruction behaviors toward 
each other [18]. In these cases, the two items were worded identically, except that one asked the 
respondent to evaluate the teaching presence behavior of the instructor, and the other asked the 
respondent to evaluate the same teaching presence behavior among other course participants. There were 
therefore six facilitating discourse items pertaining to instructors’ behaviors, and six pertaining to 
students’ behaviors. There were five direct instruction items pertaining to instructors’ behaviors, and five 
pertaining to students’ behaviors. 

All 28 items loaded successfully on their intended constructs. This showed some support that the 
Teaching Presence Scale does measure the teaching presence construct as intended in a professional 
development setting. The stronger loadings were all on the items designed to measure only the teaching 
presence behaviors of the facilitator, while the weaker ones were on the items designed to measure 
teaching presence behaviors among the participants. In terms of model fit, fit indices for the three-factor, 
28-item model were found to be contradictory and therefore somewhat weak. Thus, analysis of the 
original teaching presence model showed some support for a three-factor model, but it did not show solid 
evidence that the model is generalizable to the population at large. Because of the lack of such evidence, 
and because of the existence of two other models in the literature, ancillary analyses were conducted to 
re-assess the factor structure, factor loadings, and model fit. 

A. Ancillary Analysis #1: A Three-Factor Model with 17 Items 

The first ancillary model explored was based on one proposed by Shea et al. [28] and validated by 
Arbaugh and Hwang [20]. In both of these studies, the authors found evidence to support the elimination 
of 6 items from the analysis of the facilitating discourse factor and 5 items from the analysis of the direct 
instruction factor. These items were the ones whose wording focused on the teaching presence behaviors 
of the participants. The present study also found the items pertaining to participants’ teaching presence 
behaviors had factor loadings consistently lower than those of the items pertaining to facilitators’ teaching 
presence behaviors. Thus, the CFA was re-run after eliminating all of the items designed to measure 
participants’ behaviors. The results of this change to 17 facilitator-focused variables showed a substantial 
improvement in model fit. As with the initial analysis, all items loaded successfully on the three factors 
they were designed to measure. With the elimination of the weaker, participant-focused variables, r 2 
values for the factor loadings were all within .81 to .93. This was an improvement of 25% on the lower 
end of the r 2 value range. Additionally, fit indices improved to show almost universal support for the 
model’s goodness-of-fit. This ancillary analysis provided stronger support for each of the first five 
hypotheses, as the model fit improved for the three-factor model, the items (with those pertaining to 
participants’ behaviors eliminated) were clearly shown to be valid measures of the teaching presence 
construct, and the items (with those pertaining to participants’ behaviors eliminated) loading strongly on 
their proposed factors. 

The improvement brought about by the elimination of items pertaining to participants’ behaviors is 
interesting in light of the role that peers are expected to play in the learning process, as proposed by such 
learning theorists as Vygotsky, Piaget and Bandura [8, 9, 10], Vygotsky [8] argued that people learn 
through collaboration with peers (e.g., classmates) or superiors (e.g., teachers) possessing greater levels of 
competence, with the more competent assisting the less competent through the learning process. Piaget 
also acknowledged the role of peer interaction in learning, and Bandura concluded that people tend to 
learn largely through observation and “cognitively active” imitation of social role models [10, p. 64]. 
Bandura also advocated a model of bidirectional reciprocation in learning, replacing behaviorist models 
based on “unidirectional stimulus-response connections” and placing emphasis on the importance of 
community interactions and mutual learning experiences [10, p. 65]. 

While the two previously-described CFAs of the TPS show that the facilitator-oriented items are strong 
and valid measures of teaching presence, the participant-oriented items were much weaker representations 
of the construct in this study. This could indicate that peer collaboration is a stronger part of the learning 
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process as described by the other parts of the Community of Inquiry (social presence and cognitive 
presence). 

Social presence and cognitive presence are, by definition, the components of the Community of Inquiry 
model that describe student-to-student behaviors and interactions. Social presence is defined by Garrison 
et al. as “the ability of participants in a Community of Inquiry to project themselves socially and 
emotionally, as ‘real’ people (i.e., their full personality), through the medium of communication being 
used,” particularly in the medium of computer-mediated communication [14, p. 94], Cognitive presence is 
defined as “the extent to which learners are able to construct and confirm meaning through sustained 
reflection and discourse in a critical Community of Inquiry” [14, p. 11]. The results of the current CFA 
seem to indicate that participant behaviors should be left within these two student-oriented components, 
rather than examined within the teaching presence component. 

The weakness of the participant-oriented TPS items compared to the facilitator-oriented items may also 
shed light on student perceptions of the role of the facilitator (i.e., that the facilitator is at the center of the 
learning process and that students are recipients of knowledge rather than co-instructors). Although the 
role of instructor in an online educational setting is ideally one of guide and collaborative member of the 
learning team, rather than oracle and lecturer [38], students still seem to perceive a reasonably strong 
characterization of the instructor or facilitator as the figure in charge of the learning process. This 
perception seems to have held true across educational settings, from undergraduate higher education [17, 
18, 19, 28] to graduate higher education [20] to professional development at FOR-PD of adult students 
possessing at least a bachelor’s degree. 

It may seem understandable that in-service teachers like those at FOR-PD would see the instructor or 
facilitator as the true leader of the learning process. These participants coidd have a teacher-centered 
perspective on the learning process because they are all educators themselves. However, the participants 
in Arbaugh and Hwang’s [20] study were students in a graduate business management education 
program, and the participants in the various studies conducted at the SUNY Learning Network were 
undergraduate students in a variety of different programs of study ([17, 18, 19, 28], The participants in 
these studies were not pulled from populations of teachers like those from FOR-PD. It may be, then, that 
the common thread among all of the participants in these teaching presence studies is that their view of 
instructor behaviors stems from their own schooling experiences, in which traditional pedagogy 
emphasized teacher-centered over student-centered learning environments. Thus, it could be that even 
with the recommendation of student-centered approaches for optimal learning [14, 17, 18, 19, 28, 38, 39, 
40, 41], there are societal norms that still place the teacher at the center of the learning process; even with 
student-centered approaches being considered ideal for online and adult education, real-life practice still 
keeps the teacher as the dominant force in learning and interactions. 

The analysis of this 17-item, three-factor model indicated much stronger goodness-of-fit than the original 
28-item, three-factor model. These criteria are important because they give an indication of 
generalizability of the results to the larger population outside of that studied, by comparing the fit of the 
model in the study to that of a perfect, saturated model. These fit indices offer support to the idea that 
teaching presence, which is a model based upon good, general principles of learning, is as useful for 
describing phenomena in online professional development as it is for describing those in optimal online 
higher educational experiences. Certainly, the continued use of a three-factor structure of teaching 
presence, including instructional design and organization, facilitating discourse, and direct instruction, 
was supported. 

B. Ancillary Analysis #2: A Two-Factor Model with 17 Items 

The second ancillary analysis tested the two-factor model of teaching presence proposed by Shea et al. 
[28] after exploratory factor analysis conducted on TPS showed the 17 items loading on two factors, 
instead of three. These two factors were called (a) instructional design and organization; and (b) directed 
facilitation. Item 6, which was originally written to be a measure of instructional design and organization 
[17], loaded on the directed facilitation factor instead [28]. As with the first ancillary analysis, the model 
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fit indices for the analysis of this two-factor model show a much better fit than the initial analysis of the 
three-factor, 28-item model. While very close to the goodness-of-fit shown in the first ancillary analysis, 
these fit indices are slightly weaker. In all, the results of both ancillary analyses were stronger than those 
of the initial analysis. This provides solid support for the elimination of the participant-focused items. 
And while the indicators did load with statistical significance on the two-factor model, both the slight 
decrease in model fit and the fact that one item (#6) seemed not to fit as well on its new factor (directed 
facilitation) made the two-factor model a poorer fit than the 17-item three-factor model. This supports the 
results of the analysis performed by Arbaugh and Hwang [20] more so than those of the analysis 
performed by Shea et al., [28], Garrison [21] proposed that the discrepancies in model fit (two-factor 
versus three-factor) could possibly be explained by the differences in study participants between the two, 
with those in the analysis by Shea et al., [28] being undergraduate students, and those in the analysis by 
Arbaugh and Hwang [20] being graduate students with more experience in higher educational settings. 
Garrison [21] suggested that graduate students might have a better-developed ability to observe subtle 
distinctions in the behavior patterns of their online instructors, leading to their support of the three-factor 
model. 

It is also possible that older students like those in the study by Arbaugh and Hwang [20] and those in the 
current study at FOR-PD possess greater concentrations of the characteristics of adult learners discussed 
earlier [42, 43, 44]. It is possible the additional life experience and/or work experience they have gained 
since completing their undergraduate education, plus the additional maturity that may develop as students 
get older, may make graduate students a substantively different group of learners than their younger, less 
experienced counterparts in undergraduate online education. This makes sense, intuitively, as many 
undergraduate students are transitioning from the pedagogy they have known throughout primary and 
secondary school to the andragogical principles that are supposed to be most effective in online education 
[42]. 

In traditional pedagogy, the student is viewed as a dependent personality and the instructor’s role is to 
build upon the student’s experience. Traditional pedagogy regards students’ readiness to learn as being 
developmentally-based and age-based. It emphasizes a subject-centered orientation to learning, and 
extrinsic motivation to learn, based on a system of rewards and punishment. In andragogy, by contrast, 
the emphasis is on the student as increasingly self-directed. The instructor’s role is to tap into the learner’s 
existing life experience as a rich resource for the entire learning community. Students’ readiness to learn 
is seen as being based upon life work and problems rather than age and development. A task-centered or 
problem-centered orientation to learning takes the place of the subject-orientation found in traditional 
pedagogy, and motivation to learn is viewed as intrinsic—i.e., based upon internal curiosity or the need to 
solve problems [42], 

It is possible students in the graduate sample and the professional development sample have made the 
transition from traditional pedagogy to the andragogy seen in online learning experiences more 
completely than undergraduate students, leading to the different results obtained in the three analyses. The 
more experienced students may have different perspectives and perceptions of the role of the instructor 
than do undergraduates. 

C. Relationship Between Teaching Presence and Student Satisfaction 

Correlations were generated to determine the relationship between the 17-item three teaching presence 
composite variables (instructional design and organization, facilitating discourse, and direct instruction) 
and the two student satisfaction composite variables (satisfaction with facilitators and satisfaction with the 
course). For the most part, the results indicated positive and moderate to strong relationships between 
these variables. As a participant’s sense of teaching presence moves in a positive direction, so does 
his/her feeling of satisfaction with both the course and the facilitator. Again, this supports the 
appropriateness of the application of principles of andragogy to a learner population made up of adult 
professionals [42, 43], Not only did the students in this study rate their instructors’ teaching presence 
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behaviors quite highly, their reported sense of satisfaction with the facilitators and the course was also 
quite high and was highly correlated with their acknowledgment of teaching presence behaviors. 

The teaching presence-student satisfaction relationships were strongest between direct instruction and 
facilitator satisfaction, followed by facilitating discourse and facilitator satisfaction, and instructional 
design and organization with facilitator satisfaction. It is notable that facilitating discourse and direct 
instruction, for which Shea et al. [29] found evidence to support creating one combined factor (directed 
facilitation), had the strongest correlations with student satisfaction. In the current study, there was little 
ambiguity about students’ ability to recognize and appreciate the two distinct factors. This could be 
attributable to the differences between the characteristics and motivations of the different study samples 
[ 21 ]. 

There were also moderate to strong correlations between direct instruction and course satisfaction, 
followed by facilitating discourse and course satisfaction, and finally by instructional design and 
organization and course satisfaction. Although the correlations between teaching presence and student 
satisfaction with the course were slightly weaker, they followed the same pattern with direct instruction 
having the greatest impact, then facilitating discourse, then instructional design and organization. Both in 
terms of the participants’ perceptions of their facilitators and of their experience with the course, direct 
instruction had the strongest influence on students’ feelings of satisfaction. When considering the 
principles of effective adult online education proposed by Gibbons and Wentworth [42], it would seem 
that in spite of the self-directedness, independence, and intrinsic motivation shared by these students, they 
still place great value on the leadership of a high-quality instructor/facilitator. 

It also seems that participants in the online professional development setting in this study saw a clear 
distinction between the roles of their facilitator and the roles of other participants. Even though both 
participants and facilitators are all technically professional peers from the same industry, the facilitators 
were valued most for being directors and leaders of the learning experience, more than for facilitating 
discourse (guiding and collaborating in the learning interactions) or for their instructional design and 
organization capabilities. Participants expected their facilitator to be a teacher, leader, and mentor first, a 
referee of interpersonal communications second, and a curriculum designer and maintainer third. In spite 
of the theoretical value of collaboration with peers throughout the learning process [8, 9, 10], there was 
evidence to support the notion that the participants in this study placed more value on guidance from their 
facilitators than from their fellow students. 

VI. LIMITATIONS 

This study only includes participants who were enrolled in and completed the FOR-PD course during the 
spring of 2009. It was assumed that non-completers chose to leave the course for a variety of reasons, the 
full range of which is unknown. As non-completers of a course do not have the same course experience 
(at least in terms of duration and completeness, and possibly in terms of other affective factors as well), 
they were eliminated from the study in the interest of working with the most homogeneous sample as 
possible. 

Additionally, the TPS relies on self-report. As indicated, this self-report is given only by participants who 
are nearing completion of the FOR-PD course. Because respondents are successful completers of the 
course, by nature of the end-of-course survey administration, data from less successful or less satisfied 
(non-completing) participants are lacking from the analysis. This may present a view of teaching presence 
that is not generalizable to non-completers. Additionally, while the response rate was quite high which 
increases the likelihood of generalization, the perceptions of those responding may differ from those that 
did not respond. 

As discussed previously, the TPS relies on learner perceptions of teaching presence instead of direct 
observations of teaching behaviors. One disadvantage of evaluating online learning experiences using 
learner perceptions is the rather shallow nature of the data obtained. If one considers Kirkpatrick’s four 
levels of evaluation, level one is reaction (e.g., data from student satisfaction surveys), level two is 
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learning (e.g., evaluation of students’ grades, pre- and posttests for learning gains, or rubrics for analysis 
of student course work and interactions), level three is behavior (e.g., observation of students’ application 
of what they have learned to real life), and level four is results (e.g., measuring ways the students’ 
learning gains have generated results at an organizational level) [22], Higher-level applications of 
learning are more difficult and expensive to evaluate, which helps account for the prevalence of level one 
evaluations, i.e., student reports of course satisfaction [22], 

The problem with evaluations conducted at this level is that student reactions may not always be a valid 
measure of a teacher’s effectiveness within the learning experience. Shelvin, Banyard, Davies, and 
Griffiths [24] studied the use of student satisfaction surveys to evaluate university professors’ teaching 
effectiveness. They found the charisma of the instructor, as rated by students, accounted for a greater 
variance in student satisfaction than did the instructors’ perceived teaching abilities or course design. In 
other words, the positive or negative student perceptions of instructors’ personalities outweighed their 
perceptions of instructors’ teaching abilities. Yet the results of such end-of-course student satisfaction 
surveys are routinely used as measures of online and face-to-face instructors’ teaching effectiveness [24]. 
While learner perceptions are important, their objectivity may be challenged by experiences or feelings 
unrelated to the teaching presence construct being measured. 

VII. IMPLICATIONS FOR PRACTICE 

The implications for practice center mostly on how enhanced knowledge of teaching presence may be 
used to develop instructors or facilitators as online educators of adults. It is important that such instructors 
have a solid knowledge base in their field, as well as knowledge of and experience with andragogy. They 
should be taught the differences between pedagogy and andragogy, so that they clearly understand the 
teaching and learning principles that are most appropriate for students in the unique online setting. This is 
particularly true for those online instructors in professional development situations, in which learners are 
all adults who have accumulated substantial life and work experience, expect education to be relevant to 
their needs and interests, adopt a problem-solving approach toward learning, and tend to be independent, 
self-directed, intrinsically motivated learners [42, 43]. 

The results of this study also indicate it would be worthwhile to educate instructors or facilitators on the 
tenets of teaching presence so they have a model for optimal online learning interactions grounded in 
theory. In all, it would be ideal to combine explicit instruction on andragogy with explicit instruction on 
teaching presence, and perhaps the other components of the Community of Inquiry (i.e., cognitive 
presence and social presence) to optimize online learning experiences in a variety of educational settings. 

Another application resulting from this study is the development of an assessment tool for online 
educators that is not solely based on student reports. Student “course satisfaction” reports make up the 
majority of such assessments, but they have been deemed relatively shallow and unreliable as analysis 
tools [22, 23, 24], It is worthwhile to use teaching presence principles and andragogical principles to 
create a more objective assessment to be used by online program administrators, principle investigators, 
and/or instructors themselves to gauge the effectiveness of online educational experiences. 

VIII. IMPLICATIONS FOR FURTHER RESEARCH 

There are a number of ideas for future research that result from this study. First, as teaching presence and 
the instrument designed to measure it (TPS) have been shown to be appropriate for students in a 
professional development setting, it would be worthwhile to see if this extends to a wider variety of 
online educational settings (e.g., corporate, military, online secondary education). It may also be 
worthwhile to study teaching presence in conjunction with the other two components of the Community 
of Inquiry (social presence and cognitive presence). The results of this study indicated a weakness of 
participant-focused TPS items as measures of teaching presence; this could mean that they are, in fact, 
better measures of social presence or cognitive presence within the learning experience. It would be 
interesting to learn whether an instrument developed to measure all three Community of Inquiry 
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components would shed additional light on the role of peer interactions in the learning process, and how 
these interactions work in a variety of different online educational settings. Finally, it could be very 
illuminating to gauge the ability of the teaching presence construct to predict student outcomes other than 
student satisfaction (e.g., learning gains, repeat participation in other online course experiences, indication 
that students are using what they have learned in real world settings, etc.). 
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